# Distillation Learning

The first state space model to surpass Transformer-based audio classifiers, achieving state-of-the-art performance on AudioSet audio classification tasks while significantly reducing model size.

A distilled vision transformer model, pre-trained at 224x224 resolution and fine-tuned on ImageNet-1k at 384x384 resolution, learning from a teacher model via distillation tokens.

Distilled Vision Transformer model trained on Face-Mask18K dataset for efficient mask detection